This document provides an introduction and overview of DBIx::Class, an ORM (object relational mapper) for Perl. It discusses setting up tables for an authors and books example database, and performing CRUD (create, read, update, delete) operations both manually using SQL and using DBIx::Class. It also covers creating models with Schema::Loader, debugging, overloading result and result set classes, and inflating/deflating columns.
Part 1 of "Introduction to Catalyst" talks about installing Catalyst, creating your first application, examining its structure, and how HTTP requests are dispatched.
Presentation given at OSCON 2009 and PostgreSQL West 09. Describes SQL solutions to a selection of object-oriented problems:
- Extensibility
- Polymorphism
- Hierarchies
- Using ORM in MVC application architecture
These slides are excerpted from another presentation, "SQL Antipatterns Strike Back."
Part 1 of "Introduction to Catalyst" talks about installing Catalyst, creating your first application, examining its structure, and how HTTP requests are dispatched.
Presentation given at OSCON 2009 and PostgreSQL West 09. Describes SQL solutions to a selection of object-oriented problems:
- Extensibility
- Polymorphism
- Hierarchies
- Using ORM in MVC application architecture
These slides are excerpted from another presentation, "SQL Antipatterns Strike Back."
Trees In The Database - Advanced data structuresLorenzo Alberton
Storing tree structures in a bi-dimensional table has always been problematic. The simplest tree models are usually quite inefficient, while more complex ones aren't necessarily better. In this talk I briefly go through the most used models (adjacency list, materialized path, nested sets) and introduce some more advanced ones belonging to the nested intervals family (Farey algorithm, Continued Fractions, and other encodings). I describe the advantages and pitfalls of each model, some proprietary solutions (e.g. Oracle's CONNECT BY) and one of the SQL Standard's upcoming features, Common Table Expressions.
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas PatilSpark Summit
In this talk, we will discuss several advantages of the Spark RDD API for developing custom applications when compared to pure SQL-like interfaces such as Hive. In particular, we will describe how to control data distribution, avoid data skew, and implement application specific optimizations in order to build performant and reliable data pipelines. In order to illustrate these ideas, we will share our experiences redesigning a large-scale, complex (100+ stage) language model training pipeline for Spark that was originally built in Hive. The final Spark based pipeline is modular, readable, and more maintainable when compared to previous set of HQL queries. In addition to the qualitative improvements, we also observed a significant reduction in both resource usage and data landing time. Finally, we will also describe Spark optimizations that we implemented for this workload that can be applied toward batch workloads in general.
Demostración de la creación de una clase de JAVA que emplea un atributo de tipo Objeto y que al mismo tiempo se declara como vector. Se emplean los métodos GET para traer información de los objetos.
This presentation is on SQL constraints. This presentation was used by me in my YouTube vlog on SQL constraints. Vlog link: https://youtu.be/E94YFmATqb4
My Beginners Perl tutorial, much abbreviated and as presented at the London Perl Workshop on Dec 1st 2007.
Also includes the section on Regular Expressions that we didn't have time for on the day.
We all have tasks from time to time for bulk-loading external data into MySQL. What's the best way of doing this? That's the task I faced recently when I was asked to help benchmark a multi-terrabyte database. We had to find the most efficient method to reload test data repeatedly without taking days to do it each time. In my presentation, I'll show you several alternative methods for bulk data loading, and describe the practical steps to use them efficiently. I'll cover SQL scripts, the mysqlimport tool, MySQL Workbench import, the CSV storage engine, and the Memcached API. I'll also give MySQL tuning tips for data loading, and how to use multi-threaded clients.
If your not using an ORM (object relational mapper) and are still writing SQL by hand, here's what you need to know.
An introduction into DBIx::Class and some of the concepts and goodies you should be aware off.
Trees In The Database - Advanced data structuresLorenzo Alberton
Storing tree structures in a bi-dimensional table has always been problematic. The simplest tree models are usually quite inefficient, while more complex ones aren't necessarily better. In this talk I briefly go through the most used models (adjacency list, materialized path, nested sets) and introduce some more advanced ones belonging to the nested intervals family (Farey algorithm, Continued Fractions, and other encodings). I describe the advantages and pitfalls of each model, some proprietary solutions (e.g. Oracle's CONNECT BY) and one of the SQL Standard's upcoming features, Common Table Expressions.
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas PatilSpark Summit
In this talk, we will discuss several advantages of the Spark RDD API for developing custom applications when compared to pure SQL-like interfaces such as Hive. In particular, we will describe how to control data distribution, avoid data skew, and implement application specific optimizations in order to build performant and reliable data pipelines. In order to illustrate these ideas, we will share our experiences redesigning a large-scale, complex (100+ stage) language model training pipeline for Spark that was originally built in Hive. The final Spark based pipeline is modular, readable, and more maintainable when compared to previous set of HQL queries. In addition to the qualitative improvements, we also observed a significant reduction in both resource usage and data landing time. Finally, we will also describe Spark optimizations that we implemented for this workload that can be applied toward batch workloads in general.
Demostración de la creación de una clase de JAVA que emplea un atributo de tipo Objeto y que al mismo tiempo se declara como vector. Se emplean los métodos GET para traer información de los objetos.
This presentation is on SQL constraints. This presentation was used by me in my YouTube vlog on SQL constraints. Vlog link: https://youtu.be/E94YFmATqb4
My Beginners Perl tutorial, much abbreviated and as presented at the London Perl Workshop on Dec 1st 2007.
Also includes the section on Regular Expressions that we didn't have time for on the day.
We all have tasks from time to time for bulk-loading external data into MySQL. What's the best way of doing this? That's the task I faced recently when I was asked to help benchmark a multi-terrabyte database. We had to find the most efficient method to reload test data repeatedly without taking days to do it each time. In my presentation, I'll show you several alternative methods for bulk data loading, and describe the practical steps to use them efficiently. I'll cover SQL scripts, the mysqlimport tool, MySQL Workbench import, the CSV storage engine, and the Memcached API. I'll also give MySQL tuning tips for data loading, and how to use multi-threaded clients.
If your not using an ORM (object relational mapper) and are still writing SQL by hand, here's what you need to know.
An introduction into DBIx::Class and some of the concepts and goodies you should be aware off.
A presentation on how to semi-automatically build CRUD application prototypes using PHP and some PEAR libraries. This was way before Rails became popular and automatic form generation for web applications was not as common as it is now.
Also, it was the first public presentation I ever held. Ah, those were the times ;-)
Originally held at the International PHP Conference 2004 in Amsterdam, although these slides have been updated by project contributors to reflect feature additions that were implemented later.
DBIx-DataModel is an object-relational mapping framework for Perl5. Schema declarations are inspired from UML modelling. The API provides efficient interaction with the DBI layer, detailed control on statement execution steps, flexible and powerful treatment of database joins. More on http://search.cpan.org/dist/DBIx-DataModel.
Talk presented at YAPC::EU::2011 Riga (updated from a previous version presented at FPW2010).
Php 102: Out with the Bad, In with the GoodJeremy Kendall
In this session, we'll look at a typical PHP application, review a few of the horrible mistakes the fictional developer made, and then refactor the app according to some best practices. Along the way you might even learn a thing or two about PHP you don't already know.
This describes a Functional Programming approach to computing AWS Glacier "tree hash" values, hiding the tail-call elimination in Perl5 with a keyword and also shows how to accomplish the same result in Perl6.
This was the talk actually given at YAPC::NA 2016 by Dr. Conway and myself.
These are slides from our Big Data Warehouse Meetup in April. We talked about NoSQL databases: What they are, how they’re used and where they fit in existing enterprise data ecosystems.
Mike O’Brian from 10gen, introduced the syntax and usage patterns for a new aggregation system in MongoDB and give some demonstrations of aggregation using the new system. The new MongoDB aggregation framework makes it simple to do tasks such as counting, averaging, and finding minima or maxima while grouping by keys in a collection, complementing MongoDB’s built-in map/reduce capabilities.
For more information, visit our website at http://casertaconcepts.com/ or email us at info@casertaconcepts.com.
An overview of the main questions/design issues when starting to work with databases in Perl
- choosing a database
- matching DB datatypes to Perl datatypes
- DBI architecture (handles, drivers, etc.)
- steps of DBI interaction : prepare/execute/fetch
- ORM principles and difficulties, ORMs on CPAN
- a few examples with DBIx::DataModel
- performance issues
First given at YAPC::EU::2009 in Lisbon. Updated version given at FPW2011 in Paris and YAPC::EU::2011 in Riga
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
DBIx::Class beginners
1. DBIx::Class (aka DBIC)
for (advanced) beginners
Leo Lapworth @ LPW 2008
http://leo.cuckoo.org/projects/
2. assumptions
You know a little about Perl
and using objects
You know a little bit about
databases and using foreign
keys
3. DBIx::Class?
• ORM (object relational mapper)
• SQL <-> OO (using objects instead of SQL)
• Simple, powerful, complex, fab and confusing
• There are many ORMs, DBIx::Class just happens to
be the best in Perl (personal opinion)
4. why this talk?
• Help avoid mistakes I made!
• Help learn DBIx::Class faster
• Make your coding easier
5. point of note
quot;Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it.quot; - Brian W. Kernighan
This talk is about making it easy so we you are less
likely to get confused
8. authors table
CREATE TABLE authors(
id int(8) primary key auto_increment,
name varchar(255)
) engine = InnoDB DEFAULT CHARSET=utf8;
9. tips
Name tables as simple plurals (add an S) -
makes relationships easier to understand
(issue: Matt Trout quot;Tables should not be plural as gives you plurals for Result::
package names which represent a single rowquot; - talk may be rewritten in future
to reflect this as this is better once you understand the relationship setup -
either way, consistency is important)
Use a character set (UTF8) from the start
(for international characters)
23. DBIC: create
my $book = $book_model->create({
title => 'A book title',
author => $author_id,
});
Look ma, no SQL!
Tip: do not pass in primary_key field, even if its empty/undef as the
object returned will have an empty id, even if your field is auto
increment.
24. DBIC: create
my $book = $book_model->create({
title => 'A book title',
author => $author_id,
});
26. DBIC: create
my $book = $pratchett->create_related(
'books', {
title => 'Another Discworld book',
});
or
my $book = $pratchett->add_to_books({
title => 'Another Discworld book',
});
27. DBIC: create
my $book = $pratchett->create_related(
'books', {
title => 'Another Discworld book',
});
or
my $book = $pratchett->add_to_books({
title => 'Another Discworld book',
});
28. DBIC: retrieve
DBIx::Class - Lots of ways to do the same thing...
quot;There is more than one way to do it (TIMTOWTDI,
usually pronounced quot;Tim Toadyquot;) is a Perl mottoquot;
29. DBIC: retrieve
my $book = $book_model->find($book_id);
my $book = $book_model->search({
title => 'A book title',
})->single();
my @books = $book_model->search({
author => $author_id,
})->all();
30. DBIC: retrieve
while( my $book = $books_rs->next() ) {
print 'Author of '
. $book->title()
. ' is '
. $book->author()->name()
. quot;nquot;;
}
48. Result::
package LPW::DBIC::Result::Books;
use base 'DBIx::Class';
use strict;
use warnings;
sub isbn {
my $self = shift;
# search amazon or something
my $api = Amazon::API->book({
title => $self->title()
});
return $api->isbn();
}
1;
49. Result::
package LPW::DBIC::Result::Books;
use base 'DBIx::Class';
use strict;
use warnings;
sub isbn {
my $self = shift;
# search amazon or something
my $api = Amazon::API->book({
title => $self->title()
});
return $api->isbn();
}
1;
51. Result:: (inflating)
package LPW::DBIC::Result::Books;
use base 'DBIx::Class';
use strict;
use warnings;
use DateTime::Format::MySQL;
__PACKAGE__->inflate_column(
'date_published',
{ inflate => sub {
DateTime::Format::MySQL->parse_date(shift);
},
deflate => sub {
shift->ymd();
},
}
);
# Automatic see: DBIx::Class::InflateColumn::DateTime
52. Result:: (inflating)
package LPW::DBIC::Result::Books;
use base 'DBIx::Class';
use strict;
use warnings;
use DateTime::Format::MySQL;
__PACKAGE__->inflate_column(
'date_published',
{ inflate => sub {
DateTime::Format::MySQL->parse_date(shift);
},
deflate => sub {
shift->ymd();
},
}
);
# Automatic see: DBIx::Class::InflateColumn::DateTime
59. ResultSets::chaining
use LPW::DBIC;
my $book_model = LPW::DBIC->resultset('Books');
my $author_model = LPW::DBIC->resultset('Authors');
my $author = $author_model->search({
name => 'Douglas Adams',
})->single();
my $book_rs = $book_model->the_ultimate_books()
->by_author($author);
my @books = $book_rs->all();
60. ResultSets::chaining
my $book_rs = $book_model
->the_ultimate_books()
->by_author($author);
or
my $book_rs = $book_model
->the_ultimate_books();
$book_rs = $book_rs->by_author($author);
# Debug (SQL):
# SELECT me.id, me.title, me.date_published, me.author
# FROM books me
# WHERE ( ( ( author = ? ) AND ( title LIKE ? ) ) ): '5', '%42%'
61. ResultSets::chaining
my $rs = $book_model
->category('childrens')
->by_author($author)
->published_after('1812')
->first_page_contains('once upon')
->rating_greater_than(4);
my @books = $rs->all();
63. overloading before
new record
package LPW::DBIC::Result::Authors;
use base 'DBIx::Class';
sub new {
my ( $class, $attrs ) = @_;
# Mess with $attrs
my $new = $class->next::method($attrs);
return $new;
}
1;
72. has_many
package LPW::DBIC::Result::Books;
__PACKAGE__->has_many(
quot;author_and_booksquot;,
# Name of accessor
quot;LPW::DBIC::Result::AuthorAndBooksquot;,
# Related class
{ quot;foreign.bookquot; => quot;self.idquot; },
# Relationship (magic often works if not
# specified, but avoid!)
);
80. many_to_many
package LPW::DBIC::Result::Books;
use base 'DBIx::Class';
__PACKAGE__->many_to_many(
quot;authorsquot;
# Accessor Name
=> quot;author_and_booksquot;,
# has_many accessor_name
'author'
# foreign relationship name
);
1;
81. many_to_many
package LPW::DBIC::Result::Authors;
use base 'DBIx::Class';
__PACKAGE__->many_to_many(
quot;booksquot;
# Accessor Name
=> quot;author_and_booksquot;,
# has_many accessor_name
'book'
# foreign relationship name
);
1;
# This is NOT auto generated by Schema::Loader
82. using many_to_many
#!/usr/bin/perl
use LPW::DBIC;
my $author_model = LPW::DBIC->resultset('Authors');
my $author = $author_model->search({
name => 'Douglas Adams',
})->single();
$author->add_to_books({
title => 'A new book',
});
83. using many_to_many
my $author = $author_model->search({
name => 'Douglas Adams',
})->single();
$author->add_to_books({
title => 'A new book',
});
# SELECT me.id, me.name FROM authors me
# WHERE ( name = ? ): 'Douglas Adams';
# INSERT INTO books (title) VALUES (?): 'A new book';
# INSERT INTO author_and_books (author, book)
# VALUES (?, ?): '5', '2';
87. error messages
DBIx::Class::Schema::Loader::connection
(): Failed to load external class
definition for
'LPW::DBIC::Result::Authors': Can't
locate object method quot;many_to_manyquot; via
package quot;LPW::DBIC::Result::Authorquot; at
lib/LPW/DBIC/Result/Authors.pm line 9.
Compilation failed in require at /
Library/Perl/5.8.8/DBIx/Class/Schema/
Loader/Base.pm line 292.
88. error messages
DBIx::Class::Schema::Loader::connection
(): Failed to load external class
definition for
'LPW::DBIC::Result::Authors': Can't
locate object method quot;many_to_manyquot; via
package quot;LPW::DBIC::Result::Authorquot; at
lib/LPW/DBIC/Result/Authors.pm line 9.
Compilation failed in require at /
Library/Perl/5.8.8/DBIx/Class/Schema/
Loader/Base.pm line 292.
89. errors
• Turn on debugging
• Read error messages (sometimes useful!)
• Check field names
• Check package names
• Check which database you are connected
to (development/test/live?) - repeat above
91. Template Toolkit
• [% author.books.count %] not working?
• TT all methods are called in list context
• [% author.books_rs.count %] scalar context
Available for all relationships
93. Catalyst
package Your::App::Model::LPW;
use base qw(Catalyst::Model::DBIC::Schema);
use strict;
use warnings;
__PACKAGE__->config(
schema_class => 'LPW::DBIC',
);
1;
Keep your Scheme in a separate
package to your Catalyst application
94. Catalyst
sub action_name : Local {
my ($self, $c) = @_;
my $model = $c->model('DBIC::LPW');
my $author_model = $model->resultset('Authors');
}
1;